1 link tagged with all of: residual stream + interpretability + multilingual + representation geometry + llm
Click any tag below to further narrow down your results
Links
The author builds an interactive atlas of Qwen3-1.7B's residual stream, tracking how about 1,000 question-and-answer examples across four languages and Python land in the model's internal vector space. The result is a view of language and topic structure as overlapping geometric sheets.
- The axes track language strongly, but topic structure persists after accounting for language, so the model encodes both what is discussed and how it is said.
- Code examples trace paths through partly shared regions with natural-language examples, suggesting a common geometric structure across very different text.
- The headline 17.6% figure is token-state variance under the Global PCA axes, not a measure of meaning or reasoning, and the author cautions that the sample is small, templated, and single-turn rather than real multi-turn conversation.